Automatic sound synthesizer programming: techniques and applications

The aim of this thesis is to investigate techniques for, and applications of automatic sound synthesizer programming. An automatic sound synthesizer programmer is a system which removes the requirement to explicitly specify parameter settings for a sound synthesis algorithm from the user. Two forms of these systems are discussed in this thesis: tone matching programmers and synthesis space explorers. A tone matching programmer takes at its input a sound synthesis algorithm and a desired target sound. At its output it produces a configuration for the sound synthesis algorithm which causes it to emit a similar sound to the target. The techniques for achieving this that are investigated are genetic algorithms, neural networks, hill climbers and data driven approaches. A synthesis space explorer provides a user with a representation of the space of possible sounds that a synthesizer can produce and allows them to interactively explore this space. The applications of automatic sound synthesizer programming that are investigated include studio tools, an autonomous musical agent and a self-reprogramming drum machine. The research employs several methodologies: the development of novel software frameworks and tools, the examination of existing software at the source code and performance levels and user trials of the tools and software. The main contributions made are: a method for visualisation of sound synthesis space and low dimensional control of sound synthesizers; a general purpose framework for the deployment and testing of sound synthesis and optimisation algorithms in the SuperCollider language sclang; a comparison of a variety of optimisation techniques for sound synthesizer programming; an analysis of sound synthesizer error surfaces; a general purpose sound synthesizer programmer compatible with industry standard tools; an automatic improviser which passes a loose equivalent of the Turing test for Jazz musicians, i.e. being half of a man-machine duet which was rated as one of the best sessions of 2009 on the BBC's 'Jazz on 3' programme

Studio report: sound synthesis with DDSP and network bending techniques

This paper reports on our experiences synthesizing sounds and building network bending functionality onto the Differentiable Digital Signal Processing (DDSP) system. DDSP is an extension to the TensorFlow API with which we can embed trainable signal processing nodes in neural networks. Comparing DDSP sound synthesis networks to preset finding networks and sample level synthesis networks, we argue that it offers a third mode of working, providing continuous control in real-time of high fidelity synthesizers using low numbers of control parameters. We describe two phases of our experimentation. Firstly we worked with a composer to explore different training datasets and parameters. Secondly, we extended DDSP models with network bending functionality, which allows us to feed additional control data into the network's hidden layers and achieve new timbral effects. We describe several possible network bending techniques and how they affect the sound

Social machines for education driven by feedback agents

The aim of this paper is to explain some of the ways in which multi agent system (MAS) theory can be used to describe, design and enhance social machines (also referred to as Socio-Cognitive Systems). We believe there is a really opportunity for the MAS community to engage with emerging theory and practice of designing such systems. Social machines - also referred to as Socio-Cognitive Systems from the MAS community - are terms used to refer to the recent breed of technological systems which allow human and computational agents to socially interact, typically on a large scale and sometimes towards achieving shared goals. Examples include social networking platforms and crowd sourced encyclopaedias. The discussion of social machines and MAS is taken from three perspectives. Firstly, the theoretical notion of an abstract social machine as a socio-cognitive system containing humans and agents is introduced. Secondly, a speci#12;c instance of a social machine which has been designed to enable social music learning supported by agents is described. Thirdly, an agent architecture which is designed for operation within educational social machines is discussed, with particular focus on what we believe is the core currency of these machines: feedback.Much of of this work was undertaken as part of the FP7 project in the Technology Enhanced Learning Program called Practice and Performance Analysis Inspiring Social Education (PRAISE) involving the 1st and 2nd authors. We acknowledge Harry Brenton, Marco Gillies Andreu Grimalt-Reynes, Jonathan James, Edgar Jones, Julian Padget and Harko Harko Verhagen who have helped in discussions. The second author received support from the European Network for Social Intelligence, SINTELNET (FET Open Coordinated Action FP7-ICT-2009-C Project No. 286370) for short term visits to the IIIA to work with the 3rd author.Peer Reviewe

Progress report on the EAVI BCI toolkit for music: musical applications of algorithms for use with consumer brain computer interfaces

No description supplie

Beyond Motivation and Engagement: Students’ Voices on the Use of Game-Based Learning in a Bachelor Computer Science Online Degree

Video games have many potential uses beyond pure entertainment, including their use in educational contexts. Yet, it remains really challenging to put together guidelines to design effective game-like interventions in educational contexts. This study examines existing work relating to gamification, game-based learning, and serious games, and finds there is still limited qualitative work concerning the student perspective and limited work developing pedagogical guidelines for developers wishing to develop effective game-based learning experiences. The study focuses on the perception of students in regard to game-based learning activities in the context of a BSc Computer Science online degree. Students enrolled in the online degree were invited to fill in an online survey after their experience with a selection of game-based learning activities in the online degree. Reflexive Thematic Analysis was used to evaluate the open-ended responses from 55 participants. First, quantitative and qualitative results revealed insightful information along with four overarching themes (“Complementary to lectures on topics that are usually hard or too abstract to teach”, “Allow students to take on identities and learn from different angles and perspectives”, “Balanced challenge and context relevance to minimise students wasting their time”, and “Reward players for their effort with meaningful rewards and provide a safe space for failure”), suggesting that game-based learning interventions offer more than just motivation and engagement. Second, technical and pedagogical principles emerged from the data analysis, proposing guidelines for future designers of game-based learning activities in similar educational contexts. Finally, the study provides a selection of twelve open-source and browser-based game-based learning activities, the ones students encountered in the BSc Computer Science online degree

The pop song generator: designing an online course to teach collaborative, creative AI

This article describes and evaluates a new online AI-creativity course. The course is based around three near-state-of-the-art AI models combined into a pop song generating system. A fine-tuned GPT-2 model writes lyrics, Music-VAE composes musical scores and instrumentation and Diffsinger synthesises a singing voice. We explain the decisions made in designing the course which is based on Piagetian, constructivist 'learning-by-doing'. We present details of the five-week course design with learning objectives, technical concepts, and creative and technical activities. We explain how we overcame technical challenges to build a complete pop song generator system, consisting of Python scripts, pre-trained models, and Javascript code that runs in a dockerised Linux container via a web-based IDE. A quantitative analysis of student activity provides evidence on engagement and a benchmark for future improvements. A qualitative analysis of a workshop with experts validated the overall course design, it suggested the need for a stronger creative brief and ethical and legal content

Sun Begins to Fade

A score written by Max de Wardener for the Call and Response projec

Pedagogical Agents for Social Music Learning in Crowd-Based Socio-Cognitive Systems

This paper considers some of the issues involved in building a crowdbased system for learning music socially in communities. The effective implementation of building such systems provides several fascinating challenges if they are to be sufficiently flexible and personal for effective social learning to take place when they are large number of users. Based on our experiences of building the infrastructure for a crowd-based music learning system in Goldsmiths called MusicCircle we address several some of the challenges using an agent based approach, employing formal specifications to articulate the agent design which can later be used for software development. The challenges addressed are: 1) How can a learner be provided with a personalised learning experience? 2) How can a learner make best use of the heterogenous community of humans and agents who co-habit the virtual learning environment? We present formal specifications for an open learner model, a learning environment, learning plans and a personal learning agent. The open learner model represents the learner as having current and desired skills and knowledge and past and present learning plans. The learning environment is an online platform affording learning tasks which can be carried out by individuals or communities of users and agents. Tasks are connected together into learning plans, with pre and post conditions. We demonstrate how the personal learning agent can find learning plans and propose social connections for its user within a system which affords a dynamic set of learning plans and a range of human/agent social relationships, such as learner teacher, learner-learner and producer-commentator

A Comparison of Parametric Optimisation Techniques for Musical Instrument Tone Matching

Parametric optimisation techniques are compared in their abilities to elicit parameter settings for sound synthesis algorithms which cause them to emit sounds as similar as possible to target sounds. A hill climber, a genetic algorithm, a neural net and a data driven approach are compared. The error metric used is the Euclidean distance in MFCC feature space. This metric is justified on the basis of its success in previous work. The genetic algorithm offers the best results with the FM and subtractive test synthesizers but the hill climber and data driven approach also offer strong performance. The concept of sound synthesis error surfaces, allowing the detailed description of sound synthesis space, is introduced. The error surface for an FM synthesizer is described and suggestions are made as to the resolution required to effectively represent these surfaces. This information is used to inform future plans for algorithm improvements